Model Selection for Regularized Least-Squares Algorithm in Learning Theory

We investigate the problem of model selection for learning algorithms depending on a continuous parameter. We propose a model selection procedure based on a worst case analysis and data-independent choice of the parameter. For regularized least-squares algorithm we bound the generalization error of the solution by a quantity depending on few known constants and we show that the corresponding model selection procedure reduces to solving a bias-variance problem. Under suitable smoothness condition on the regression function, we estimate the optimal parameter as function of the number of data and we prove that this choice ensures consistency of the algorithm.